Systems for determining location using robots with deformable sensors

ABSTRACT

Systems and methods for determining a location of a robot are provided. A method includes receiving, by a processor, a signal from a deformable sensor including data with respect to a deformation region in a deformable membrane of the deformable sensor resulting from contact with a first object. The data associated with contact with the first object is compared, by the processor, to details associated with contact with the first object to information associated with a plurality of objects stored in a database. The first object is identified, by the processor, as a first identified object of the plurality of objects stored in the database. The first identified object is an object of the plurality of objects stored in the database that is most similar to the first object. The location of the robot is determined, by the processor, based on a location of the first identified object.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to co-pending U.S. Provisional PatentApplication No. 62/977,466, filed Feb. 17, 2020, for “Systems ForDetermining Location Using Robots With Bubble Sensors,” and co-pendingU.S. Provisional Patent Application No. 62/984,083, filed on Mar. 2,2020, for “Bubble Sensor Grippers For Robust Manipulation And ManipulandState Estimation,” which are hereby incorporated by reference in theirentirety including the drawings.

TECHNICAL FIELD

Embodiments described herein generally relate to systems and methods fordetermining a location of a robot within a space and, more particularly,robots having deformable contact and geometry/pose sensors capable ofdetecting contact and a geometry of an object to determine a location ofthe robot.

BACKGROUND

As humans, our sense of touch allows us to determine the shape of anobject without looking at the object. This assists us in identifying theobject and, without using our sense of sight, we may be able toascertain where we are located within a space based on our knowledge ofthe location of the objected touched.

Robots are commonly equipped with end effectors that are configured toperform certain tasks. However, robots do not have varying levels oftouch sensitivity as do humans. End effectors may include sensors suchas pressure sensors, but such sensors provide limited information aboutthe object that is in contact with the end effector. Thus, the robot maydamage a target object by using too much force, or drop the objectbecause it does not properly grasp the object. As such, in someapplications, a deformable/compliant end effector may be desirable.

However, robots are currently not capable of contacting an object inthis manner and, as a result, the robots are not capable of identifyingthe object. Therefore, to determine a position or location of the robot,robots rely on other technology, such as GPS sensors or visual sensors,to identify the location of the robot. However, these may turn out to beinaccurate in small-scale environments.

SUMMARY

In one embodiment, a method for determining a location of a robotincluding a deformable sensor includes receiving, by a processor, asignal from a deformable sensor including data with respect to adeformation region in a deformable membrane of the deformable sensorresulting from contact with a first object. The data associated withcontact with the first object is compared, by the processor, to detailsassociated with contact with the first object to information associatedwith a plurality of objects stored in a database. The first object isidentified, by the processor, as a first identified object of theplurality of objects stored in the database. The first identified objectis an object of the plurality of objects stored in the database that ismost similar to the first object. The location of the robot isdetermined, by the processor, based on a location of the firstidentified object.

In another embodiment, a robot for determining a location within a spaceincludes a casing including an upper surface, an opposite lower surface,and an edge surface extending between the upper surface and the lowersurface. At least one deformable sensor is provided on the casing. Theat least one deformable sensor includes an internal sensor and adeformable membrane. The internal sensor is configured to output adeformation region with the deformable membrane as a result of contactwith a first object. The robot includes one or more processors and oneor more memory modules including a computer-readable medium storingcomputer-readable instructions that, when executed by the one or moreprocessors, cause the one or more processors to receive data from theinternal sensor representing the deformation region when the firstobject is contacted. The data associated with contact of the firstobject is compared by the processor to details associated with aplurality of objects stored in a database. The first identified objectis an object of the plurality of objects stored in the database that ismost similar to the first object. The processor identifies the firstobject as a first identified object of the plurality of objects storedin the database. A location of the robot is determined by the processorbased on a location of the first identified object.

In yet another embodiment, a system for determining a location of arobot including a deformable sensor includes a robot including an uppersurface, an opposite lower surface, and an edge surface extendingbetween the upper surface and the lower surface. At least one deformablesensor is provided on the robot. The at least one deformable sensorincludes a housing, a deformable membrane coupled to an upper portion ofthe housing, an enclosure configured to be filled with a medium, and aninternal sensor disposed within the enclosure having a field of viewconfigured to be directed through the medium and toward a bottom surfaceof the deformable membrane. The internal sensor is configured to outputa deformation region within the deformable membrane as a result ofcontact with a first object. The system includes one or more processorsand one or more memory modules including a computer-readable mediumstoring computer-readable instructions that, when executed by the one ormore processors, cause the one or more processors to receive data fromthe internal sensor representing the deformation region when the firstobject is contacted. The data associated with contact of the firstobject is compared by the processor to details associated with aplurality of objects stored in a database. The first identified objectis an object of the plurality of objects stored in the database that ismost similar to the first object. The processor identifies the firstobject as a first identified object of the plurality of objects storedin the database. A location of the robot is determined by the processorbased on a location of the first identified object.

These and additional features provided by the embodiments describedherein will be more fully understood in view of the following detaileddescription, in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments set forth in the drawings are illustrative and exemplaryin nature and not intended to limit the subject matter defined by theclaims. The following detailed description of the illustrativeembodiments can be understood when read in conjunction with thefollowing drawings, where like structure is indicated with likereference numerals and in which:

FIG. 1 schematically depicts a perspective view of an example robotincluding a plurality of deformable sensors according to one or moreembodiments described and illustrated herein;

FIG. 2 is a block diagram illustrating hardware utilized by the robot ofFIG. 1 for implementing various processes and systems according one ormore embodiments described and illustrated herein;

FIG. 3 schematically depicts a cross-sectional view of an exampledeformable sensor according to one or more embodiments described andillustrated herein;

FIG. 4 schematically depicts a top perspective view of the exampledeformable sensor of FIG. 3 according to one or more embodimentsdescribed and illustrated herein;

FIG. 5 schematically depicts a cross-sectional view of an exampledeformable sensor according to one or more embodiments described andillustrated herein;

FIG. 6 schematically depicts a rear perspective view of a bubble moduleof the example deformable sensor of FIG. 5 according to one or moreembodiments described and illustrated herein;

FIG. 7 schematically depicts an exploded view of the bubble sensor ofFIG. 6 according to one or more embodiments described and illustratedherein;

FIG. 8 schematically depicts a filter layer coupled to a deformablemembrane of a deformable sensor according to one or more embodimentsdescribed and illustrated herein;

FIG. 9 schematically depicts a filter within a field of view of aninternal sensor of a deformable sensor according to one or moreembodiments described and illustrated herein;

FIG. 10 schematically depicts a dot pattern on a bottom surface of adeformable membrane of a deformable sensor according to one or moreembodiments described and illustrated herein;

FIG. 11 schematically depicts a grid pattern on a bottom surface of adeformable membrane of a deformable sensor according to one or moreembodiments described and illustrated herein;

FIG. 12 schematically depicts a compound internal sensor having aplurality of internal sensors according to one or more embodimentsdescribed and illustrated herein;

FIG. 13 is an image depicting an output of a deformable sensor on anelectronic display according to one or more embodiments described andillustrated herein;

FIG. 14 is a flow chart depicting an exemplary process of determiningthe pose and force associated with an object in contact with adeformable sensor according to one or more embodiments described andillustrated herein;

FIG. 15 schematically depicts an overhead view of a space in which therobot is utilized and performs an operation according to one or moreembodiments described and illustrated herein; and

FIG. 16 is a flow chart depicting an exemplary process of determining alocation of the robot within a space according to one or moreembodiments described and illustrated herein.

DETAILED DESCRIPTION

Embodiments of the present disclosure are directed to robots includingdeformable sensors and, more particularly, deformable/compliant contactand/or geometry sensors (hereinafter “deformable sensors”) that not onlydetect contact with a target object, but also detect the geometry, pose,and contact force of the target object to identify a location of therobots. Particularly, the deformable sensors described herein comprise adeformable membrane coupled to a housing that maintains a sensor capableof detecting displacement of the deformable membrane by contact with anobject. Thus, the deformable sensors described herein provide a robot(or other device) with a sense of touch when manipulating or contactingobjects.

Autonomous robots are used for accomplishing various tasks and mayefficiently navigate a space, such as a building or an individual room.Such tasks may include retrieving an item, delivering an item, or, inthe case of an autonomous vacuum, cleaning a room. Known autonomousvacuums map a space, such as a room of a house, by driving in straightdirections until an object, such as a wall, step, or other obstacle iscontacted, and the robot stores the location information of thecontacted object in its memory as an object to be avoided during futurecleaning operations. As additional objects are contacted, the robotcontinually adds location information of these objects to its memory tomore accurately map the room and avoid these objects in the future.

As shown in FIG. 1, an example robot 100 is illustrated as an autonomousvacuum and the robot operation referred to herein is a cleaningoperation. However, it should be appreciated that the robot 100 may beany other suitable robot other than an autonomous vacuum for performingother tasks without departing from the scope of the present disclosure.For example, the robot 100 may be an object retrieval or delivery robotinstructed to navigate a space between a starting point and adestination point.

The robot 100 generally includes a casing 102 defined by an uppersurface 104, an opposite lower surface 106, and an edge surface 108extending between the upper surface 104 and the lower surface 106. Thecasing 102 houses the internal components of the robot 100, describedherein. The lower surface 106 of the casing 102 faces a downwarddirection toward a floor surface F and the upper surface 104 faces in anopposite upward direction. It should be understood that embodiments arenot limited to the casing 102 configuration of FIG. 1, and the varioussurfaces may take on other shapes. At least one wheel 110 is provided onthe lower surface 106 of the casing 102 to permit the robot 100 totraverse the floor surface F. In some embodiments, the robot 100includes a pair of wheels 110. In other embodiments, the robot 100 mayinclude legs including joints, skis, rails, or flying components formoving or transporting the robot 100.

Further, the robot 100 includes at least one deformable sensor 112provided on the edge surface 108 of the casing 102. In some embodiments,the robot 100 includes a plurality of deformable sensors 112 spacedapart from one another along the edge surface 108 of the casing 102. Inembodiments in which a plurality of deformable sensors 112 are provided,the plurality of deformable sensors 112 may be spaced apart and arrangedalong the edge surface 108 of the casing 102 in any suitable manner.However, in some embodiments, the deformable sensors 112 may be locatedon the upper surface 104 and/or the lower surface 106 of the casing 102.When the deformable sensors 112 are located on the lower surface 106 ofthe casing 102, the deformable sensors 112 may be suitable for sensingthe type of floor the robot 100 is rolling over, the height a threshold,or the like. In other embodiments, the robot 100 may include any numberof arms including joints with an end effector attached to an end of thearm. In this embodiment, the deformable sensors 112 may be provided onthe end effector and independently movable with respect to the robot100.

The deformable sensors 112 provided on the robot 100 may be any suitabledeformable sensors, such as those embodiments discussed herein, capableof identifying characteristics, such as geometry, pose, hardness,flexibility, and the like, of an object contacted. The ensuingdescription of the robot 100 will refer to the deformable sensors 112generally with regard to their use in identifying an object andassisting the robot 100 in determining its location.

Referring now to FIG. 2, example components of one non-limitingembodiment of the robot 100 is schematically depicted. In someembodiments, the robot 100 includes a communication path 228, aprocessor 230, a memory module 232, an inertial measurement unit 236, aninput device 238, a camera 244, network interface hardware 246, alocation sensor 250, a light 252, a proximity sensor 254, a motorizedwheel assembly 258, a battery 260, and a charging port 262. Thecomponents of the robot 100 may be contained within or mounted to thecasing 102. The various components of the robot 100 and the interactionthereof will be described in detail below.

Still referring to FIG. 2, the communication path 228 may be formed fromany medium that is capable of transmitting a signal such as, forexample, conductive wires, conductive traces, optical waveguides, or thelike. Moreover, the communication path 228 may be formed from acombination of mediums capable of transmitting signals. In oneembodiment, the communication path 228 comprises a combination ofconductive traces, conductive wires, connectors, and buses thatcooperate to permit the transmission of electrical data signals tocomponents such as processors, memories, sensors, input devices, outputdevices, and communication devices. Accordingly, the communication path228 may comprise a bus. Additionally, it is noted that the term “signal”means a waveform (e.g., electrical, optical, magnetic, mechanical orelectromagnetic), such as DC, AC, sinusoidal-wave, triangular-wave,square-wave, vibration, and the like, capable of traveling through amedium. The communication path 228 communicatively couples the variouscomponents of the robot 100. As used herein, the term “communicativelycoupled” means that coupled components are capable of exchanging datasignals with one another such as, for example, electrical signals viaconductive medium, electromagnetic signals via air, optical signals viaoptical waveguides, and the like.

The processor 230 of the robot 100 may be any device capable ofexecuting computer-readable instructions. Accordingly, the processor 230may be a controller, an integrated circuit, a microchip, a computer, orany other computing device. The processor 230 may be communicativelycoupled to the other components of the robot 100 by the communicationpath 228. This may, in various embodiments, allow the processor 230 toreceive data from the one or more deformable sensors 112. In otherembodiments, the processor 230 may receive data directly from one ormore internal sensors, which are part of one or more deformable sensors112 on a robot 100. Accordingly, the communication path 228 maycommunicatively couple any number of processors with one another, andallow the components coupled to the communication path 228 to operate ina distributed computing environment. Specifically, each of thecomponents may operate as a node that may send and/or receive data.While the embodiment depicted in FIG. 2 includes a single processor 230,other embodiments may include more than one processor.

Still referring to FIG. 2, the memory module 232 of the robot 100 iscoupled to the communication path 228 and communicatively coupled to theprocessor 230. The memory module 232 may, for example, containcomputer-readable instructions to detect a shape of an object that hasdeformed the deformable sensors 112. In this example, these instructionsstored in the memory module 232, when executed by the processor 230, mayallow for the determination of the shape of an object based on theobserved deformation of the deformable sensors 112. The memory module232 may comprise RAM, ROM, flash memories, hard drives, or anynon-transitory memory device capable of storing computer-readableinstructions such that the computer-readable instructions can beaccessed and executed by the processor 230. The computer-readableinstructions may comprise logic or algorithm(s) written in anyprogramming language of any generation (e.g., 1GL, 2GL, 3GL, 4GL, or5GL) such as, for example, machine language that may be directlyexecuted by the processor, or assembly language, object-orientedprogramming (OOP), scripting languages, microcode, etc., that may becompiled or assembled into computer-readable instructions and stored inthe memory module 232. Alternatively, the computer-readable instructionsmay be written in a hardware description language (HDL), such as logicimplemented via either a field-programmable gate array (FPGA)configuration or an application-specific integrated circuit (ASIC), ortheir equivalents. Accordingly, the functionality described herein maybe implemented in any conventional computer programming language, aspre-programmed hardware elements, or as a combination of hardware andsoftware components. While the embodiment depicted in FIG. 2 includes asingle memory module 232, other embodiments may include more than onememory module.

The inertial measurement unit 236, if provided, is coupled to thecommunication path 228 and communicatively coupled to the processor 230.The inertial measurement unit 236 may include one or more accelerometersand one or more gyroscopes. The inertial measurement unit 236 transformssensed physical movement of the robot 100 into a signal indicative of anorientation, a rotation, a velocity, or an acceleration of the robot100. The operation of the robot 100 may depend on an orientation of therobot 100 (e.g., whether the robot 100 is horizontal, tilted, or thelike). Some embodiments of the robot 100 may not include the inertialmeasurement unit 236, such as embodiments that include an accelerometerbut not a gyroscope, embodiments that include a gyroscope but not anaccelerometer, or embodiments that include neither an accelerometer nora gyroscope.

One or more input devices 238 are coupled to the communication path 228and communicatively coupled to the processor 230. The input device 238may be any device capable of transforming user contact into a datasignal that can be transmitted over the communication path 228 such as,for example, a button, a switch, a knob, a microphone or the like. Invarious embodiments, an input device 238 may be the deformable sensor112 as described herein. In some embodiments, the input device 238includes a power button, a volume button, an activation button, a scrollbutton, or the like. The one or more input devices 238 may be providedso that the user may interact with the robot 100, such as to navigatemenus, make selections, set preferences, and other functionalitydescribed herein. In some embodiments, the input device 238 includes apressure sensor, a touch-sensitive region, a pressure strip, or thelike. It should be understood that some embodiments may not include theinput device 238. As described in more detail below, embodiments of therobot 100 may include multiple input devices disposed on any surface ofthe casing 102. In some embodiments, one or more of the input devices238 are configured as a fingerprint sensor for unlocking the robot 100.For example, only a user with a registered fingerprint may unlock anduse the robot 100.

The camera 244 is coupled to the communication path 228 andcommunicatively coupled to the processor 230. The camera 244 may be anydevice having an array of sensing devices (e.g., pixels) capable ofdetecting radiation in an ultraviolet wavelength band, a visible lightwavelength band, or an infrared wavelength band. The camera 244 may haveany resolution. The camera 244 may be an omni-directional camera, or apanoramic camera. In some embodiments, one or more optical components,such as a mirror, fish-eye lens, or any other type of lens may beoptically coupled to the camera 244. As described in more detail below,the camera 244 is a component of an imaging assembly 222 operable to beraised above the casing 102 to capture image data.

The network interface hardware 246 is coupled to the communication path228 and communicatively coupled to the processor 230. The networkinterface hardware 246 may be any device capable of transmitting and/orreceiving data via a network 270. Accordingly, network interfacehardware 246 can include a wireless communication module configured as acommunication transceiver for sending and/or receiving any wired orwireless communication. For example, the network interface hardware 246may include an antenna, a modem, LAN port, Wi-Fi card, WiMax card,mobile communications hardware, near-field communication hardware,satellite communication hardware and/or any wired or wireless hardwarefor communicating with other networks and/or devices. In one embodiment,network interface hardware 246 includes hardware configured to operatein accordance with the Bluetooth wireless communication protocol. Inanother embodiment, network interface hardware 246 may include aBluetooth send/receive module for sending and receiving Bluetoothcommunications to/from a portable electronic device 280. The networkinterface hardware 246 may also include a radio frequency identification(“RFID”) reader configured to interrogate and read RFID tags.

In some embodiments, the robot 100 may be communicatively coupled to aportable electronic device 280 via the network 270. In some embodiments,the network 270 is a personal area network that utilizes Bluetoothtechnology to communicatively couple the robot 100 and the portableelectronic device 280. In other embodiments, the network 270 may includeone or more computer networks (e.g., a personal area network, a localarea network, or a wide area network), cellular networks, satellitenetworks and/or a global positioning system and combinations thereof.Accordingly, the robot 100 can be communicatively coupled to the network270 via wires, via a wide area network, via a local area network, via apersonal area network, via a cellular network, via a satellite network,or the like. Suitable local area networks may include wired Ethernetand/or wireless technologies such as, for example, wireless fidelity(Wi-Fi). Suitable personal area networks may include wirelesstechnologies such as, for example, IrDA, Bluetooth, Wireless USB,Z-Wave, ZigBee, and/or other near field communication protocols.Suitable personal area networks may similarly include wired computerbuses such as, for example, USB and FireWire. Suitable cellular networksinclude, but are not limited to, technologies such as LTE, WiMAX, UMTS,CDMA, and GSM.

As stated above, the network 270 may be utilized to communicativelycouple the robot 100 with the portable electronic device 280. Theportable electronic device 280 may include a mobile phone, a smartphone,a personal digital assistant, a camera, a dedicated mobile media player,a mobile personal computer, a laptop computer, and/or any other portableelectronic device capable of being communicatively coupled with therobot 100. The portable electronic device 280 may include one or moreprocessors and one or more memories. The one or more processors canexecute logic to communicate with the robot 100. The portable electronicdevice 280 may be configured with wired and/or wireless communicationfunctionality for communicating with the robot 100. In some embodiments,the portable electronic device 280 may perform one or more elements ofthe functionality described herein, such as in embodiments in which thefunctionality described herein is distributed between the robot 100 andthe portable electronic device 280.

The location sensor 250 is coupled to the communication path 228 andcommunicatively coupled to the processor 230. The location sensor 250may be any device capable of generating an output indicative of alocation. In some embodiments, the location sensor 250 includes a globalpositioning system (GPS) sensor, though embodiments are not limitedthereto. Some embodiments may not include the location sensor 250, suchas embodiments in which the robot 100 does not determine a location ofthe robot 100 or embodiments in which the location is determined inother ways (e.g., based on information received from the camera 244, thenetwork interface hardware 246, the proximity sensor 254, the inertialmeasurement unit 236 or the like). The location sensor 250 may also beconfigured as a wireless signal sensor capable of triangulating alocation of the robot 100 and the user by way of wireless signalsreceived from one or more wireless signal antennas.

The motorized wheel assembly 258 is coupled to the communication path228 and communicatively coupled to the processor 230. As described inmore detail below, the motorized wheel assembly 258 includes the atleast one wheel 110 driven by one or more motors (not shown). Theprocessor 230 may provide one or more drive signals to the motorizedwheel assembly 258 to actuate the wheels 110 such that the robot 100travels to a desired location, such as a location that the user wishesto acquire environmental information (e.g., the location of particularobjects within at or near the desired location).

The light 252, if provided, is coupled to the communication path 228 andcommunicatively coupled to the processor 230. The light 252 may be anydevice capable of outputting light, such as, but not limited to, a lightemitting diode, an incandescent light, a fluorescent light, or the like.Some embodiments include a power indicator light that is illuminatedwhen the robot 100 is powered on. Some embodiments include an activityindicator light that is illuminated when the robot 100 is active orprocessing data. Some embodiments include an illumination light forilluminating the environment in which the robot 100 is located. Someembodiments may not include the light 252.

The proximity sensor 254, if provided, is coupled to the communicationpath 228 and communicatively coupled to the processor 230. The proximitysensor 254 may be any device capable of outputting a proximity signalindicative of a proximity of the robot 100 to another object. In someembodiments, the proximity sensor 254 may include a laser scanner, acapacitive displacement sensor, a Doppler effect sensor, an eddy-currentsensor, an ultrasonic sensor, a magnetic sensor, an internal sensor, aradar sensor, a LiDAR sensor, a sonar sensor, or the like. Someembodiments may not include the proximity sensor 254, such asembodiments in which the proximity of the robot 100 to an object isdetermined from inputs provided by other sensors (e.g., the camera 244,etc.) or embodiments that do not determine a proximity of the robot 100to an object.

The robot 100 may be powered by the battery 260, which is electricallycoupled to the various electrical components of the robot 100. Thebattery 260 may be any device capable of storing electric energy forlater use by the robot 100. In some embodiments, the battery 260 is arechargeable battery, such as a lithium-ion battery or a nickel-cadmiumbattery. In embodiments in which the battery 260 is a rechargeablebattery, the robot 100 may include the charging port 262, which may beused to charge the battery 260. Some embodiments may not include thebattery 260, such as embodiments in which the robot 100 is powered theelectrical grid, by solar energy, or by energy harvested from theenvironment. Some embodiments may not include the charging port 262,such as embodiments in which the robot 100 utilizes disposable batteriesfor power.

Referring now to FIGS. 3 and 4, an embodiment of the deformable sensor112, of the robot 100 is schematically illustrated. FIG. 3 is across-sectional view of the example deformable sensor 112 and FIG. 4 isa top perspective view of the example deformable sensor 112. The exampledeformable sensor 112 generally comprises a housing 310 and a deformablemembrane 320 coupled to the housing 310, such as by an upper portion 311of the housing 310. In some embodiments, the housing 310 is 3D printed.The housing 310 and the deformable membrane 320 define an enclosure 313that is filled with a medium through one or more fluid conduits 312,which may be a valve or any other suitable mechanism. The fluid conduit312 may be utilized to fill or empty the enclosure 313. In one example,the medium is gas, such as air. Thus, air may be pumped into theenclosure 313 to a desired pressure such that the deformable membrane320 forms a dome shape as shown in FIGS. 3 and 4, although any suitableshape may be utilized in other embodiments. In another example, themedium is a gel, such as silicone or other rubber-like substance. Insome embodiments, a substance such as solid silicone may be cast in agiven shape before assembly of the deformable sensor 112. In variousembodiments, the medium may be anything that is transparent to aninternal sensor 330, discussed in more detail herein, such as to awavelength of a time-of-flight sensor. The medium may includeclear/transparent rubbers in some embodiments. In other embodiments, themedium may be a liquid. In some examples, the deformable membrane 320and the medium within the enclosure 313 may be fabricated of the samematerial, such as, without limitation, silicone. In some embodiments,the deformable sensor 112 may be mountable. For example, the enclosure313 may include brackets to be mounted any suitable object, such as therobot 100 or material. The deformable membrane 320 may be a latex or anyother suitable material, such as a suitably thin, non-porous,rubber-like material. In some embodiments, the deformable membrane 320is laser-cut from a 0.04 mm thick latex sheet.

As used herein, the term “deformability” may refer, for example, to easeof deformation of a deformable sensor. Deformability may refer to howeasily a deformable membrane deforms when contacting a target object.The deformability of the deformable sensor 112 may be tuned/modified bychanging the material of the deformable membrane 320 and/or the pressurewithin the enclosure 313. By using a softer material (e.g., softsilicone), the deformable sensor 112 may be more easily deformed.Similarly, lowering the pressure within the enclosure 313 may also causethe deformable membrane 320 to more easily deform, which may in turnprovide for a more deformable sensor 112. In some embodiments, thedeformable membrane 320 is inflated to a height of 20 mm to 75 mm and toa pressure of 0.20 psi to 0.30 psi.

As used herein, the term “spatial resolution” may refer, for example, tohow many pixels a deformable sensor has. The number of pixels may rangefrom 1 (e.g., a sensor that simply detects contact with a target object)to thousands or millions (e.g., the dense sensor provided by atime-of-flight sensor having thousands of pixels) or any suitablenumber. The deformable sensor 112 may be of a high spatial resolution,with a dense tactile sensing sensor that is provided as an end effectorof the robot 100, thereby giving the robot 100 a fine sense of touchlike a human's fingers. The deformable sensor 112 may also have a depthresolution to measure movement toward and away from the sensor. In someembodiments, the deformable sensor 112 features varying touchsensitivity due to varying spatial resolution and/or depth resolution.

An internal sensor 330 capable of sensing depth may be disposed withinthe enclosure 313, which may be measured by the depth resolution of theinternal sensor 330. The internal sensor 330 may have a field of view332 directed through the medium and toward a bottom surface of thedeformable membrane 320. In some embodiments, the field of view 332 ofthe internal sensor 330 is 62°×45°+/−10%. In some embodiments, theinternal sensor 330 may be an optical sensor. As described in moredetail below, the internal sensor 330 may be capable of detectingdeflections of the deformable membrane 320 when the deformable membrane320 comes into contact with an object. In one example, the internalsensor 330 is a time-of-flight sensor capable of measuring depth. Thetime-of-flight sensor emits an optical signal (e.g., an infrared signal)and has individual detectors (i.e., “pixels”) that detect how long ittakes for the reflected signal to return to the sensor. Thetime-of-flight sensor may have any desired spatial resolution. Thegreater the number of pixels, the greater the spatial resolution. Thespatial resolution of the sensor disposed within the internal sensor 330may be changed. In some cases, low spatial resolution (e.g., one “pixel”that detects a single point's displacement) may be desired. In others, asensitive time-of-flight sensor such may be used as a high spatialresolution internal sensor 330 that provides dense tactile sensing.Thus, the internal sensor 330 may be modular because the sensors may bechanged depending on the application. A non-limiting example of atime-of-flight sensor is the Pico Flexx sold by PMD Technologies AG ofSiegen, Germany. Other types of visual internal sensors include, by wayof non-limiting example, stereo cameras, laser range sensors, structuredlight sensors/3D scanners, single cameras (such as with dots or otherpatterns inside), or any other suitable type of visual detector. Forexample, the internal sensor 330 may be configured as a stereo-cameracapable of detecting deflections of the deformable membrane 320 by anobject.

Any suitable quantity and/or types of internal sensors 330 may beutilized within a single deformable sensor 112 in some embodiments. Insome examples, not all internal sensors 330 within a deformable sensor112 need be of the same type. In various embodiments, one deformablesensor 112 may utilize a single internal sensor 330 with a high spatialresolution, whereas another deformable sensor 112 may use a plurality ofinternal sensors 330 that each have a low spatial resolution. In someembodiments, the spatial resolution of a deformable sensor 112 may beincreased due to an increase in the quantity of internal sensors 330. Insome examples, a decrease in the number of internal sensors 330 within adeformable sensor 112 can be compensated for by a corresponding increasein the spatial resolution of at least some of the remaining internalsensors 330. As discussed in more detail below, the aggregatedeformation resolution may be measured as a function of the deformationresolution or depth resolution among the deformable sensors 112 in aportion of the robot 100. In some embodiments, aggregate deformationresolution may be based upon a quantity of deformable sensors 112 in aportion of the robot 100 and a deformation resolution obtained from eachdeformable sensor 112 in that portion.

Referring again to FIG. 3, a power conduit 314 may be utilized in theenclosure 313 to provide power and/or data/signals, such as to theinternal sensor 330 by way of a cable, such as for USB (universal serialbus) or any other suitable type of power and/or signal/data connection.As used herein, an airtight conduit may include any type of passagewaythrough which air or any other fluid (such as liquid) cannot pass. Inthis example, the power conduit 314 is airtight and may provide apassageway through which solid object (such as wires/cables) may passthrough with an airtight seal, such as an O-ring, being formed aroundsuch wires/cables at each end of the power conduit 314. Otherembodiments utilize wireless internal sensors 330 to transmit and/orreceive data and/or power. In various embodiments where the medium isnot a gas, such as silicone, the enclosure 313 and/or power conduit 314may not necessarily be airtight.

In some embodiments, the internal sensor 330 may include one or moreinternal pressure sensors (barometers, pressure sensors, etc., or anycombination thereof) utilized to detect the general deformation of thedeformable membrane 320 through the medium. In some embodiments, thedeformable sensor 112 and/or internal sensor 330 may receive/sendvarious data, such as through the power conduit 314 discussed above,wireless data transmission (Wi-Fi, Bluetooth, etc.), or any othersuitable data communication protocol. For example, pressure within adeformable sensor 112 may be specified by a pressurization parameter andmay be inversely proportional to the deformability of the deformablesensor 112. In some embodiments, the deformability of a deformablesensor 112 may be modified by changing pressure within the enclosure 313or a material of the deformable membrane 320. In some embodiments,receipt of an updated parameter value may result in a real-time ordelayed update (pressurization, etc.).

Referring now to FIGS. 5-7, another example deformable sensor 112′ isschematically illustrated. FIG. 5 depicts a cross-sectional view of thedeformable sensor 112′. The deformable sensor 112′ is similar to thedeformable sensor 112 illustrated in FIGS. 3 and 4, and generallycomprises a housing 410 and a bubble module 411 coupled to the housing410. The bubble module 411 includes a deformable membrane 420, similarto deformable membrane 320 shown in FIGS. 3 and 4. As such, thedeformable membrane 420 may include any of the features disclosed hereinwith respect to the deformable membrane 320. The bubble module 411 ofthe deformable sensor 112′ is similar to the upper portion 311 of thedeformable sensor 112 shown in FIGS. 3 and 4. However, the bubble module411 is removable from the housing 410 and, thus, replaceable whennecessary. The bubble module 411 defines an enclosure 413 that is filledwith a medium through one or more fluid conduits 412, which may be avalve or any other suitable mechanism, extending through the housing 410and terminating at the bubble module 411. As shown, the fluid conduit412 includes a tube 412A and a tube fitting 412B. The fluid conduit 412may be utilized to fill or empty the enclosure 413. As the enclosure 413is filled with the medium, the deformable membrane 420 forms a domeshape, as shown in FIGS. 5 and 6.

An internal sensor 430, similar to the internal sensor 330, capable ofsensing depth may be disposed within the housing 410, which may bemeasured by the depth resolution of the internal sensor 430. Theinternal sensor 430 may have a field of view 432, having an angle A1,directed through the medium and toward a bottom surface of thedeformable membrane 420. As a non-limiting example, the angle Al of thefield of view 432 of the internal sensor 430 is 62°×45°+/−10%. In someembodiments, the internal sensor 430 may be an optical sensor. Asdescribed in more detail below, the internal sensor 430 may be capableof detecting deflections of the deformable membrane 420 when thedeformable membrane 420 comes into contact with an object. In oneexample, the internal sensor 430 is a time-of-flight sensor capable ofmeasuring depth. The time-of-flight sensor emits an optical signal(e.g., an infrared signal) and has individual detectors (i.e., “pixels”)that detect how long it takes for the reflected signal to return to thesensor.

As shown in FIG. 5, the internal sensor 430 is provided within thehousing 410 and oriented at an angle A2 with respect to the bubblemodule 411 and the deformable membrane 420. Specifically, the internalsensor 430 extends along an axis with the angle A2 extending between theaxis of the internal sensor 430 and a backing plate 422 of the bubblemodule 411, discussed in more detail herein. As a non-limiting example,the angle A2 between the internal sensor 430 and the bubble module 411may be 35°+/−10%. The internal sensor 430 being angled maximizes thefield of view 432 and depth measurement accuracy at a center and distaledge of the deformable membrane 420 opposite the internal sensor 430,while minimizing an overall width dimension of the deformable sensor112′.

Referring now to FIGS. 6 and 7, the bubble module 411 of the deformablesensor 112′ is shown apart from the housing 410. As shown in FIG. 6, thebubble module 411 is shown in its assembled form, while FIG. 7illustrates an exploded view of the bubble module 411. The bubble module411 includes the deformable membrane 420, the backing plate 422, and aring 424 for securing the deformable membrane 420 onto the backing plate422. The bubble module 411 may be removably coupled to the housing 410using any suitable means, such as threaded inserts 425 extending throughholes 427 in the backing plate 422 for securing the backing plate 422 tothe housing 410. Alternatively, or in addition thereto, the threadedinserts 425 may be used to further secure an outer edge 421 of thedeformable membrane 420 to the backing plate 422.

More particularly, the backing plate 422 includes a housing surface422A, a membrane surface 422B, and an edge surface 422C extendingbetween the housing surface 422A and the membrane surface 422B. Thebacking plate 422 is formed from a transparent material, such as anacrylic, so that the field of view 432 of the internal sensor 430 is notobstructed by the bubble module 411. In assembling the bubble module411, an adhesive may be applied onto the edge surface 422C of thebacking plate 422. Thereafter, the outer edge 421 of the deformablemembrane 420 may be positioned around the backing plate 422 to contactthe edge surface 422C thereof and be adhered thereto. Further, the ring424 may be positioned around the edge surface 422C of the backing plate422 in order to sandwich the deformable membrane 420 between the backingplate 422 and the ring 424. As noted above, the threaded inserts 425 maybe used to further secure the deformable membrane 420 to the backingplate 422 by positioning the outer edge 421 of the deformable membrane420 along the housing surface 422A of the backing plate 422 andinserting the threaded inserts 425 through the outer edge 421 of thedeformable membrane 420 and the backing plate 422. As shown, the tubefitting 412B is shown attached to the backing plate 422 at an orifice423 and the tube 412A extends from the tube fitting 412B to deliver amedium into the bubble module 411.

Thus, if the deformable sensor 112′ is damaged, for example if thedeformable membrane 420 punctured, such that medium leaks out of thebubble module 411, the deformable sensor 112′ may be repaired withoutinterfering with the housing 410 and electrical components providedtherein, such as the internal sensor 430. In doing so, the bubble module411 is removed from the housing 410 via the threaded inserts 425, or anyother suitable means provided, and a replacement bubble module 411 maybe coupled to the housing 410. Alternatively, it may be desirable torepair the existing bubble module 411 by replacing only the deformablemembrane 420 or repairing the deformable membrane 420 itself byproviding a patch to seal the puncture or other damaged area. It shouldappreciated that providing the deformable sensor 112′ having the bubblemodule 411 that may be easily replaced allows for a greater portion ofthe deformable sensor 112′ to be housed within the robot 100 while onlythe bubble module 411 is exposed and accessible from an exterior of therobot 100. This reduces the size of such the robot 100 and reduces thelikelihood of damage to the deformable sensor 112′ during operation.

Referring now to FIG. 8, in some embodiments, the deformable sensors112, 112′ may include an optional filter layer 323. In a non-limitingexample, the filter layer 323 is illustrated as being provided on thedeformable sensor 112. The filter layer 323 may be disposed on a bottomsurface 321 of the deformable membrane 320. As described in more detailherein, the bottom surface 321 of the deformable membrane 320 may bepatterned (e.g., a dot pattern, a grid pattern, or any other suitabletype pattern). By way of non-limiting example, a stereo-camera may beutilized to detect displacement of the deformable membrane 320 based onidentified deformations of the patterned bottom surface 321. The filterlayer 323 may be configured to aid the internal sensor 330 in detectingdeformation of the deformable membrane 320. In some embodiments, thefilter layer 323 reduces glare or improper reflections of one or moreoptical signals emitted by the internal sensor 330. In some embodiments,the filter layer 323 may scatter one or more optical signals emitted bythe internal sensor 330. The filter layer 323 may be an additional layersecured to the bottom surface 321 of the deformable membrane 320, or itmay be a coating and/or pattern applied to the bottom surface 321 of thedeformable membrane 320.

Referring to FIG. 9, in some embodiments, the deformable sensors 112,112′ may include an internal sensor filter 335. In a non-limitingexample, the internal sensor filter 335 is illustrated as being providedon the internal sensor 330 of the deformable sensor 112. The internalsensor filter 335 may be disposed within the field of view 332 of theinternal sensor 330. The internal sensor filter 335 may optimize theoptical signal emitted by the internal sensor 330 for reflection uponthe bottom surface 321 of the deformable membrane 320. Like the filterlayer 323, the internal sensor filter 335 may be disposed within a fieldof view 332 of the internal sensor 330 and may reduce glare or improperreflections of any optical signals emitted by the internal sensor 330.In some embodiments, the internal sensor filter 335 may scatter one ormore optical signals emitted by the internal sensor 330. In someembodiments, both the filter layer 323 and the internal sensor filter335 may be utilized.

A pattern may be provided on either the bottom surface of the deformablemembrane 320 of the deformable sensor 112 or the bottom surface of thedeformable membrane 420 of the deformable sensor 112′. Referring againto FIG. 10, in a non-limiting example, a dot pattern 325 including aplurality of arranged dots may be applied to the bottom surface 321 ofthe deformable membrane 320 on the optional filter layer 323 or thedeformable membrane 320 itself to assist in the detection of thedeformation of the deformable membrane 320. For example, the dot pattern325 may assist in the detection of the deformation when the internalsensor 330 is a stereo-camera. Alternatively, a stereo-camera may beprovided in addition to the internal sensor 330 to supplement thedeformation detection of the internal sensor 330. Varying degrees ofdistortion to the dot pattern 325 may be utilized to discern how muchdeformation has occurred to the deformable membrane 320. The pattern onthe bottom surface 321 may be random and not necessarily arranged in adot pattern 325 or an array as shown in FIG. 10.

In some embodiments in which the dot pattern 325 is provided, an initialor pre-deformation image of the dot pattern 325 on the bottom surface321 of the deformable membrane 320 may be captured prior to anydeformation of the deformable membrane 320. Thereafter, the internalsensor 330, or separate stereo-camera, if provided, captures at leastone post-deformation image of the dot pattern 325 during or afterdeformation of the deformable membrane 320. The pre-deformation imagemay be compared to the post-deformation image and the location of eachdot in the pre-deformation image is compared to corresponding dots inthe post-deformation image to determine an amount of displacement of thedots and, thus, the displacement of the deformation membrane 320. Thedisplacement of each dot may be used to determine the amount ofdeformation at individual quadrants or sections of the dot pattern 325.The amount of displacement of each dot is then converted into a distancemeasurement to determine the specific deformation of the deformablemembrane 320, or sections thereof, to discern a geometry and/or pose ofthe object deforming the deformable membrane 320.

In some embodiments, measurements between each dot, or at least some ofthe dots, of the dot pattern 325 may be stored within a memory module,such as memory module 232 (FIG. 2) of the deformable sensor 112 or anassociated processor, such as processor 230 (FIG. 2). Thus, instead ofmerely determining a geometry and/or pose of the target object, thedimensions of various sections of the target object may be determined bycalculating specific deformations between adjacent dots of the dotpattern 325. When the dot pattern 325 includes a greater number of dots,the dot pattern 325 may permit detection of deformation within smallerareas of the deformable membrane 320 as compared to when the dot pattern325 includes a fewer number of dots. In embodiments in which the dots ofthe dot pattern 325 is arranged in an array, the dots may beequidistantly spaced apart from one another or arranged in any othersuitable manner. However, in some embodiments, the distances between thedots when not equidistantly spaced from one another are stored withinthe memory module to identify the arrangement of the dots. In addition,it should be appreciated that the same technique discussed above ofcomparing the pre-deformation image to the post-deformation image may berepeated for a plurality of post-deformation images taken duringdeformation of the deformable membrane 320 to provide real-time data asto the geometry, measurements, and/or pose of the target object. Bycomparing post-deformation images to one another, displacement of thedeformable membrane 320 occurring within smaller increments of time canbe determined, as opposed to a total deformation of the deformablemembrane 320 from an initial, pre-deformed state.

Referring to FIG. 11, in some embodiments, the pattern may be a gridpattern 322 applied to a bottom surface 321 of the deformable membrane320 to assist in the detection of the deformation of the deformablemembrane 320. For example, the grid pattern 322 may assist in thedetection of the deformation when the internal sensor 330 is astereo-camera. For example, varying degrees of distortion to the gridpattern 322 may be utilized to discern how much deformation hasoccurred. In this example, the distance between parallel lines and/ormeasuring curvature of lines in the grid pattern 322 may be used todetermine the amount of deformation at each point in the grid pattern322. The pattern on the bottom surface 321 may be random and notnecessarily arranged in a grid pattern 322 or an array as shown in FIG.11. It should be understood that embodiments are not limited to gridpatterns and dot patters as discussed herein, as other types of patternsare possible, such as shapes and the like.

Referring now to FIG. 12, an embodiment depicts a compound internalsensor 330′, which may be utilized instead of the internal sensor 330 ofthe deformable sensor 112 or the internal sensor 430 of the deformablesensor 112′. A plurality of internal sensors 502 are depicted, which inthis embodiment are time-of-flight cameras. Other embodiments mayutilize any combination of various types of internal sensors. In thisembodiment, cables 504 are utilized to provide data communicationsand/or power to the internal sensors, although other embodiments may usea different number of cables and/or wireless connections for data and/orpower. A support structure 506 is depicted in this embodiment, althoughother embodiments may utilize a plurality of support structures 506 orno support structure. In this embodiment, the support structure 506 isrigid, although one or more support structures 506 may be flexible tochange the orientation of internal sensors 502 in some embodiments. Inthis embodiment, the cables 504 may be connected to a base portion 508for data communications and/or power.

FIG. 13 depicts an image of an example object 615 displacing thedeformable membrane 320 of the deformable sensor 112. It should beappreciated, that the deformable sensor 112′ may also be used in thesame manner as discussed herein. In the illustrated embodiment, adisplay device 640 outputs for display on a device, output of thedeformable sensor 112 in real time as an object 615 contacts and/ordeforms the deformable membrane 320. It should be understood that thedisplay device 640 is provided for illustrative purposes only, and thatembodiments may be utilized without a display device. As the object 615is pressed into the deformable membrane 320, the object 615 imparts itsshape into the deformable membrane 320 such that the deformable membrane320 conforms to the shape of the object 615. The spatial resolution ofthe internal sensor 330 may be such that the internal sensor 330 detectsthe geometry and/or pose of the displaced deformable membrane 320. Forexample, when the internal sensor 330 is a time-of-flight sensor, theoptical signal that is reflected off of the bottom surface 321 of thedeformable membrane 320 that is being deflected by the object has ashorter time-of-flight than the optical signal that is reflected by thedeformable membrane 320 at a region outside of the deflected region.Thus, a contact region 642 (or displaced region, used hereininterchangeably) having a geometry and/or pose matching the shape of theobject 615 may be outputted and displayed on the display device 640.

The deformable sensor 112 therefore may not only detect the presence ofcontact with the object 615, but also the geometry of the object 615. Inthis manner, the robot equipped with either the deformable sensor 112 orthe deformable sensor 112′ may determine the geometry of an object basedon contact with the object. Additionally, a geometry and/or pose of theobject 615 may also be determined based on the geometric informationsensed by the deformable sensors 112, 112′. For example, a vector 644that is normal to a surface in the contact region 642 may be displayed,such as when determining the pose of the object 615. The vector 644 maybe used by a robot or other device to determine which direction aparticular object 615 may be oriented, for example.

Turning now to FIG. 14, a flowchart illustrates an exemplary method 700for determining the pose and force associated with an object in contactwith the deformable sensors 112, 112′. However, as discussed herein,reference is made to FIGS. 3 and 4 illustrating the deformable sensor112 without limiting the scope of the present disclosure. At block 702,a medium (gas, liquid, silicone, etc.) may be received within theenclosure 313 where the deformable membrane 320 is coupled to an upperportion 311 of the housing 310. At block 704, deformation of thedeformable membrane 320 may be measured based on contact with an object615 via an internal sensor 330 in the enclosure 313 having a field ofview 332 directed through the medium and toward a bottom surface 321 ofthe deformable membrane 320. At block 706, a pose of the object 615 maybe determined based on the measure deformation, such as the contactregion 642, of the deformable membrane 320. As discussed above withreference to FIG. 13, a pose of the object 615 may be determined by theobject 615 being pressed into the deformable membrane 320 and thedeformable membrane 320 conforms to the shape of the object 615.Thereafter, the internal sensor 330 detects the geometry and/or pose ofthe displaced deformable membrane 320. At block 708, an amount of forcebetween the deformable membrane 320 and the object 615 is determinedbased on the measured deformation of the deformable membrane 320. Blocks706 and 708 may be performed simultaneously, but do not necessarily needto be. At block 710, a determination is made as to whether furtherdeformation and/or contact is detected. If so, then the flowchart mayreturn to block 704. If not, the flowchart may end.

Referring now to FIG. 16, a flowchart illustrates an exemplary method900 of determining a location of the robot 100 within a space isdepicted with reference to the robot 100 illustrated in FIGS. 1 and 2and an example space 800 illustrated in FIG. 15 in which the robot 100performs an operation. As discussed herein, reference will be made tothe robot 100 utilizing the deformable sensor 112. However, the method900 is equally applicable to the robot 100 including the deformablesensor 112′ Initially, at block 902, a database is created. The databaseincludes information of at least one space and information of objectslocated within the space so that the robot 100 can determine itslocation within the space. The database may include information of aplurality of different spaces, such as different rooms in a home orfacility. Thus, the spaces stored within the database may includespecific identifiers to differentiate between the spaces, such as“Bedroom”, “Kitchen”, “Living Room”, etc. For each space, a floorplanmay be initially provided prior to an operation of the robot 100 or thefloorplan may be mapped in real-time by the robot 100 so that the robot100 can determine its location within the space. In some embodiments,the floorplan stored in the database includes dimensions of the space, alocation of objects on the floor surface of the space that the robot 100might come into contact with during its operation, dimensions of theobjects, and/or clearance distances between objects in the space.Further, the database may include specific information such as thematerial of construction of an object to associate certaincharacteristics, such as texture, hardness, flexibility, and/or thelike, with the object. For example, if an object, such as a kitchentable, is identified as being made of wood, the database will assign theobject a characteristic that the object has a specific hardnessassociated with wood, or whichever material the object is formed of.Alternatively, if an object, such as a couch, is identified as beingmade of a fabric or leather, the database will assign the object acharacteristic that the object has a hardness associated with a fabricor leather, which may be less than the hardness of the object formed ofwood. Other characteristics assigned to the objects can include theobject having rounded edges, corners, protrusions, surface features,etc. The database does not need to include each of the informationand/or characteristics discussed herein. However, it should beappreciated that the more information and characteristics that areprovided in the database with regard to the space and the objectsprovided therein, the quicker and more efficiently the robot 100 will beable to determine its location within the space.

The information of the space and/or the objects discussed herein may beentered into the database manually, such as by operating the inputdevice 238 or the portable electronic device 280. As such, theinformation of the space and/or the objects may be modified and/ordeleted as necessary when the space and/or the objects are changed, suchas when furniture is moved or replaced. The database may be stored inthe network 270 or may be stored within the memory module 232 of therobot 100 itself. However, if the database is stored in the network 270,the robot 100 uses a connection to the network 270 during operation toretrieve and utilize the information stored in the database.Alternatively, the robot 100 may be instructed to download the databaseor one or more portions of the database from the network 270 onto thememory module 232 of the robot 100 prior to performing an operation.After the robot 100 completes the operation, the database may be deletedto make additional storage space available for subsequent downloads ofthe database or portions thereof.

The deformable sensors 112 may also be utilized to map a space whencontacting an object or a wall of the space during an exploratoryoperation. This allows the robot 100 to automatically map the spaceduring an exploratory operation of the robot 100 and reduces, or in someinstances eliminates, the need for manual input by the user of theinformation of the space and the objects provided therein. However, whenmapping the space during an operation in real time, the robot 100 maynot be able to determine its location until a sufficient amount of thespace has been mapped. In some embodiments, the robot 100 may includesupplemental sensing devices in addition to the deformable sensors 112such as optical sensors, acoustic sensors, time-of-flight cameras, laserscanners, ultrasonic sensors, or the like, for automatically mapping thespace and determining information of the objects provided therein.

Referring still to the method 900 illustrated in FIG. 16, with referenceto the robot 100 illustrated in FIGS. 1 and 2 positioned within theexample space 800 illustrated in FIG. 15 and, such as a family room or aliving room in a home, and the robot 100 is activated at block 904 toperform an operation. In the case of the robot 100 being an autonomousvacuum, the operation may be a cleaning operation. In the case in whichthe robot 100 is an object retrieval robot or an object delivery robot,the operation may be navigating to a destination to retrieve or deliveran object. The example space 800 includes a plurality of objects suchas, for example, an entertainment system 802, a couch 804, a table 806,a plurality of chairs 808 arranged around the table 806, and a cabinet810. As noted above, in some embodiments, information of the space 800and the objects provided in the space 800 are manually inputted into thedatabase prior to the robot 100 beginning its operation. As such, thedatabase may include information such as the locations and/or dimensionsof the space 800, the entertainment system 802, the couch 804, the table806, the plurality of chairs 808, and the cabinet 810. As also notedabove, the database may also include the materials of construction ofeach of the objects and/or distances between each of the objects.

In an example operation as shown in FIG. 15, the robot 100 movesthroughout the space 800 and, at block 906, one of the plurality ofdeformable sensors 112 extending from the robot 100 contacts a firstobject, e.g., the cabinet 810, which may be formed of wood. Incontacting the cabinet 810, the portion of the deformable sensor 112contacting the cabinet 810 deforms. For purposes of the present exampleoperation, the first object is referred to as the cabinet 810. However,the first object may be any other object within the space 800 based onan initial starting point and travel direction of the robot 100. Asdiscussed in more detail below with regard to the deformable sensor 112,the deformable sensor 112 provides data at block 908 including at leasta geometry, pose, hardness, and/or flexibility of the cabinet 810. Thus,based on the specific deformation of the deformable sensor 112 in theillustrated example shown in FIG. 15, the robot 100 identifies that thecabinet 810 has a specific hardness. In addition, the robot 100 may alsorecognize that the deformable sensor 112 is contacting an edge or cornerof the cabinet 810 based on the specific deformation of the deformablesensor 112.

In some embodiments, the robot 100, such as the edge surface 108 of therobot 100, may contact an object at a point between adjacent deformablesensors 112 such that none of the deformable sensors 112 deform againstan object. In this case, the robot 100 may turn or rotate to repositionitself so that one of the deformable sensors 112 contact the object anddeforms. In some embodiments, the robot 100 includes additional, smallerdeformable sensors provided along the edge surface 108 of the robot 100between adjacent deformable sensors 112 to determine where the robot 100contacted the object and how to reposition the robot 100 so that one ofthe deformable sensors 112 contacts the object.

In embodiments in which the database has been populated with informationof the space 800 and the objects provided therein prior to theoperation, the robot 100 compares, at block 910, the acquiredinformation or data of the first object, e.g., the cabinet 810, which isdetermined by the deformable sensor 112, to the information associatedwith each of the objects in the space 800 stored in the database. Indoing so, the robot 100 is able to identify at block 912, or at leastnarrow the possible options, the first object in the space 800 which therobot 100 contacted. For example, the robot 100 can rule out the objectcontacted as being the couch 804 made from a fabric or leather as theassociated hardness of the couch 804 identified in the database is lessthan the associated hardness of the cabinet 810 formed of wood. Thisallows the robot 100 to determine at block 914, or at least narrow, thelocation of the robot 100 within the space 800 based on which possibleobjects in the space 800 the robot 100 may have contacted.

When the database includes a plurality of spaces, the robot 100 maycompare the first object contacted to each object within each one of thespaces in the database. However, in some embodiments, the robot 100 maycompare the first object to only those objects in a specific space orsubset of spaces based on instruction from a user indicating which spacethe robot 100 begins its operation. Alternatively, the location sensor250 of the robot 100 may be utilized to determine a general location ofthe robot 100. This may assist in narrowing the number of possiblespaces in which the robot 100 is operating.

To identify the first object as a first identified object of one of theplurality of objects in the database and, thus, the location of therobot 100, the robot 100 may also take into consideration a clearancedistance around the object and a direction traveled by the robot 100prior to contacting the first object. As shown in FIG. 15, the robot 100moves in a straight line along a distance 801 from a starting point 803prior to contacting the cabinet 810. Based on this information and theinformation associated with the space 800 provided in the database, therobot 100 may be able to rule out possible objects, such as the table806 and the chairs 808, which do not provide enough clearance to travelthe distance 801.

Once the robot 100 identifies that the first object contacted is thecabinet 810, the robot 100 can determine its location within the space800 at block 914 based on the known location of the cabinet 810 in thespace 800 stored in the database. However, if the robot 100 is notcapable of correctly identifying the first object based on the limitedamount of information acquired by the deformable sensor 112, the robot100 continues its operation until a subsequent object is contacted toconfirm the identity of the first object and the location of the robot100 from a number of possible locations within the space 800. Forexample, after contacting the first object, e.g., the cabinet 810, therobot 100 will turn away from the first object and travel in a differentdirection to continue the operation. The robot 100 will then eventuallycontact a subsequent object at block 916, such as the couch 804, andreceives a subsequent signal based on contact with the subsequentobject. At block 918, while contacting the subsequent object, e.g., thecouch 804, the deformable sensor 112 identifies the shape and hardnessof the second object in a manner similar to that which is describedherein with respect to the first object at blocks 908-912. At block 920,this information is compared to the information provided in the databaseand at block 922, the subsequent object is identified as a subsequentidentified object of one of the plurality of objects in the database, orat least the possibilities of the identity of the first object and thesubsequent object is narrowed. As noted herein, the robot 100 may alsotake into consideration a distance traveled from a point at which therobot 100 contacted the first object to a point at which the robot 100contacted the subsequent object to assist in correctly identifying thefirst object and the subsequent object.

At block 924, once the identification of the first object is confirmed,and in some instances the subsequent object if necessary, the robot 100is able to determine its location within the space 800. As such, therobot 100 may continue its operation and avoid further contact with anyother objects within the space 800 based on location information ofother objects in the space 800 provided in the database. This is usefulin instances in which the robot 100 needs to travel to a specificlocation in the space 800, such as to perform a cleaning operation or anobject retrieval/delivery operation. The location of the robot 100 mayalso be stored within the memory module 232 so that the robot 100 maycontinue to track its location within the space 800 during futureoperations. Thus, contact with additional or subsequent objects may notbe necessary. Alternatively, the robot 100 may repeat blocks 916-924with regard to a further subsequent object if the location of the robot100 is not accurately determined.

It should now be understood that embodiments of the present disclosureare directed deformable sensors capable of detecting contact with anobject as well as a geometric shape and pose of an object. One or moredeformable sensors may be provided on a robot, for example. Theinformation provided by the deformable sensors may then be used tocontrol the robot's interaction with target objects. The depthresolution and spatial resolution of the deformation sensors may varydepending on the location of the deformable sensors on the robot.

It is noted that recitations herein of a component of the presentdisclosure being “configured” or “programmed” in a particular way, toembody a particular property, or to function in a particular manner, arestructural recitations, as opposed to recitations of intended use. Morespecifically, the references herein to the manner in which a componentis “configured” or “programmed” denotes an existing physical conditionof the component and, as such, is to be taken as a definite recitationof the structural characteristics of the component.

The order of execution or performance of the operations in examples ofthe disclosure illustrated and described herein is not essential, unlessotherwise specified. That is, the operations may be performed in anyorder, unless otherwise specified, and examples of the disclosure mayinclude additional or fewer operations than those disclosed herein. Forexample, it is contemplated that executing or performing a particularoperation before, contemporaneously with, or after another operation iswithin the scope of aspects of the disclosure.

It is noted that the terms “substantially” and “about” and“approximately” may be utilized herein to represent the inherent degreeof uncertainty that may be attributed to any quantitative comparison,value, measurement, or other representation. These terms are alsoutilized herein to represent the degree by which a quantitativerepresentation may vary from a stated reference without resulting in achange in the basic function of the subject matter at issue.

While particular embodiments have been illustrated and described herein,it should be understood that various other changes and modifications maybe made without departing from the spirit and scope of the claimedsubject matter. Moreover, although various aspects of the claimedsubject matter have been described herein, such aspects need not beutilized in combination. It is therefore intended that the appendedclaims cover all such changes and modifications that are within thescope of the claimed subject matter.

What is claimed is:
 1. A method for determining a location of a robotincluding a deformable sensor, the method comprising: receiving, by aprocessor, a signal from the deformable sensor comprising data withrespect to a deformation region in a deformable membrane of thedeformable sensor resulting from contact with a first object; comparing,by the processor, the data associated with contact with the first objectto information associated with a plurality of objects stored in adatabase; identifying, by the processor, the first object as a firstidentified object of the plurality of objects stored in the database,the first identified object is an object of the plurality of objectsstored in the database that is most similar to the first object;determining, by the processor, a location of the robot based on alocation of the first identified object.
 2. The method of claim 1,further comprising: determining, by the processor, a distance traveledto the first object; identifying a clearance distance within theinformation of the plurality of objects stored in the database; andcomparing the distance traveled by the robot to the first object to theclearance distance of the plurality of objects.
 3. The method of claim1, further comprising receiving, by the processor, a subsequent signalfrom the deformable sensor comprising data with respect to thedeformation region in the deformable membrane resulting from contactwith a subsequent object.
 4. The method of claim 3, further comprisingcomparing, by the processor, the data associated with contact of thesubsequent object to information associated with the plurality ofobjects stored in the database, the information including at least oneof a geometry, a pose, a hardness, a flexibility, and a location of theplurality of objects.
 5. The method of claim 4, further comprising:identifying, by the processor, the subsequent object as a subsequentidentified object of the plurality of objects stored in the database,wherein the location of the robot is based on the location of the firstidentified object and a location of the subsequent identified object. 6.The method of claim 1, further comprising, when the robot contacts thefirst object and the deformable sensor does not contact the firstobject, repositioning the robot, by the processor, so that thedeformable sensor contacts the first object.
 7. The method of claim 1,further comprising modifying deformability of the deformable sensor bychanging a pressure within an enclosure defined in part by thedeformable membrane.
 8. The method of claim 1, further comprising:utilizing an internal sensor disposed within the deformable sensor andhaving a field of view directed through a medium stored within anenclosure of the deformable sensor and toward a bottom surface of thedeformable membrane; and scattering an optical signal emitted by theinternal sensor by a filter layer disposed on the bottom surface of thedeformable membrane.
 9. The method of claim 8, further comprisinganalyzing, by the processor, the deformation region by measuring changesto a coating or a pattern on the bottom surface of the deformablemembrane.
 10. The method of claim 8, wherein the internal sensorcomprises a time-of-flight sensor.
 11. A robot for determining alocation within a space, the robot comprising: a casing including anupper surface, an opposite lower surface, and an edge surface extendingbetween the upper surface and the lower surface; at least one deformablesensor provided on the casing, the deformable sensor comprising aninternal sensor and a deformable membrane, the internal sensorconfigured to output a deformation region within the deformable membraneas a result of contact with a first object; one or more processors; andone or more memory modules comprising a computer-readable medium storingcomputer-readable instructions that, when executed by the one or moreprocessors, cause the one or more processors to: receive data from theinternal sensor representing the deformation region of the deformablemembrane when the first object is contacted; compare the data associatedwith contact of the first object to information associated with aplurality of objects stored in a database; identify the first object asa first identified object of the plurality of objects stored in thedatabase, the first identified object is an object of the plurality ofobjects stored in the database that is most similar to the first object;and determine a location of the robot based on a location of the firstidentified object.
 12. The robot of claim 11, wherein the one or morememory modules includes computer-readable medium storingcomputer-readable instructions that, when executed by the one or moreprocessors, cause the one or more processors to: determine, by theprocessor, a distance traveled to the first object; identify a clearancedistance within the information of the plurality of objects stored inthe database; and compare the distance traveled by the robot to thefirst object to the clearance distance of the plurality of objects. 13.The robot of claim 11, wherein the one or more memory modules includescomputer-readable medium storing computer-readable instructions that,when executed by the one or more processors, cause the one or moreprocessors to: receive, by the processor, a subsequent signal from thedeformable sensor comprising data with respect to the deformation regionin the deformable membrane resulting from contact with a subsequentobject.
 14. The robot of claim 13, wherein the one or more memorymodules includes computer-readable medium storing computer-readableinstructions that, when executed by the one or more processors, causethe one or more processors to: compare, by the processor, the dataassociated with contact of the subsequent object to informationassociated with the plurality of objects stored in the database, theinformation including at least one of a geometry, a pose, a hardness, aflexibility, and a location of the plurality of objects.
 15. The robotof claim 14, wherein the one or more memory modules includescomputer-readable medium storing computer-readable instructions that,when executed by the one or more processors, cause the one or moreprocessors to: identify, by the processor, the subsequent object as asubsequent identified object of the plurality of objects stored in thedatabase, wherein the location of the robot is based on the location ofthe first identified object and a location of the subsequent identifiedobject.
 16. A system for determining a location of a robot including adeformable sensor, the system comprising: a robot including an uppersurface, an opposite lower surface, and an edge surface extendingbetween the upper surface and the lower surface; at least one deformablesensor provided on the robot, the at least one deformable sensorcomprising a housing, a deformable membrane coupled to an upper portionof the housing, an enclosure configured to be filled with a medium, andan internal sensor disposed within the housing and having a field ofview configured to be directed through the medium and toward a bottomsurface of the deformable membrane, wherein the internal sensor isconfigured to output a deformation region within the deformable membraneas a result of contact with a first object; one or more processors; andone or more memory modules comprising a computer-readable medium storingcomputer-readable instructions that, when executed by the one or moreprocessors, cause the one or more processors to: receive data from theinternal sensor representing the deformation region when the firstobject is contacted; compare the data associated with contact of thefirst object to information associated with a plurality of objectsstored in a database; identify the first object as a first identifiedobject of the plurality of objects stored in the database, the firstidentified object is an object of the plurality of objects stored in thedatabase that is most similar to the first object; and determine alocation of the robot based on a location of the first identifiedobject.
 17. The system of claim 16, wherein the one or more memorymodules includes computer-readable medium storing computer-readableinstructions that, when executed by the one or more processors, causethe one or more processors to: determine a distance traveled to thefirst object, the information of the plurality of objects stored in thedatabase including a clearance distance; and compare the distancetraveled by the robot to the first object to the clearance distance ofthe plurality of objects.
 18. The system of claim 16, wherein the one ormore memory modules includes computer-readable medium storingcomputer-readable instructions that, when executed by the one or moreprocessors, cause the one or more processors to: receive a signal fromthe deformable sensor comprising data with respect to a deformationregion in a deformable membrane resulting from contact with a subsequentobject.
 19. The system of claim 18, wherein the one or more memorymodules includes computer-readable medium storing computer-readableinstructions that, when executed by the one or more processors, causethe one or more processors to: compare the data associated with contactof the subsequent object to information associated with the plurality ofobjects stored in the database, the information including at least oneof a geometry, a pose, a hardness, a flexibility, and a location of theplurality of objects.
 20. The system of claim 19, wherein the one ormore memory modules includes computer-readable medium storingcomputer-readable instructions that, when executed by the one or moreprocessors, cause the one or more processors to: identify the subsequentobject as a subsequent identified object of the plurality of objectsstored in the database, and wherein the location of the robot is basedon the location of the first identified object and a location of thesubsequent identified object.